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CN , Abstract 

I ^ ' Analog (uncoded) transmission provides a simple and robust scheme for communicating a Gaussian 

P\J ' source over a Gaussian channel under the mean squared error (MSE) distortion measure. Unfortunately, 

its performance is usually inferior to the all-digital, separation-based source-channel coding solution, 
which requires exact knowledge of the channel at the encoder. The loss comes from the fact that except 
Y^ • for very special cases, e.g. white source and channel of matching bandwidth (BW), it is impossible to 

achieve perfect matching of source to channel and channel to source by linear means. We show that 
by combining prediction and modulo-lattice operations, it is possible to match any colored Gaussian 



(N 
> 



^^ . source to any colored Gaussian noise channel (of possibly different BW), hence achieve Shannon's 

OO ' optimum attainable performance R{D) = C. Furthermore, when the source and channel BWs are equal 

• ' (but otherwise their spectra are arbitrary), this scheme is asymptotically robust in the sense that for 

^D I high signal-to-noise ratio a single encoder (independent of the noise variance) achieves the optimum 



o 



X 



performance. The derivation is based upon a recent modulo-lattice modulation scheme for transmitting 
a Wyner-Ziv source over a dirty-paper channel. 
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I. Introduction 

Digital transmission of analog sources relies, at least from a theoretical point of view, on Shannon's 
source-channel separation principle. Being both optimal and easy to implement, digital techniques replace 
today traditional analog communication even in areas like voice telephony, radio and television. This trend 
ignores, however, the fact that the separation principle does not hold for communication networks, and 
in particular for broadcast channels and compound channels IS, |[37l . |[29l . Indeed, due to both practical 
and theoretical reasons, joint source-channel coding and hybrid digital-analog schemes are constantly 
receiving attention of researchers in the academia and the industry. 

In this work we consider transmission under the mean-squared error (MSE) distortion criterion, of 
a general stationary Gaussian source over a power-constrained channel with inter-symbol interference 
(ISI), i.e. the transmitted signal is passed through some linear filter, and additive white Gaussian noise 
(AWGN). H 

Shannon's joint source-channel coding theorem implies that the optimal (i.e., minimum distortion) 
performance D°p* is given by 

R (L>°P*) = C, (1) 

where R{D) is the rate-distortion function of the source at MSE distortion D, and C = C{P) is the 
channel capacity at power-constraint P, both given by the well-known water-filling solutions |6]. By 
Shannon's separation principle, ([T]) can be achieved by a system consisting of source and channel coding 
schemes. This system usually requires large delay and complex digital codes. An additional serious 
drawback of the all-digital system is that it suffers from a "threshold effect": if the channel noise turns 
out to be higher than expected, then the reconstruction will suffer from very large distortion, while if the 
channel has lower noise than expected, then there is no improvement in the distortion |[37l . ||29l . Ell . 

In contrast, analog communication techniques (like amplitude or frequency modulation |5|) are not 
sensitive to exact channel knowledge at the transmitter. Moreover, in spite of their low complexity and 
delay, they are sometimes optimal: if we are allowed one channel use per source sample, and the source 
and noise are white (i.e. have i.i.d. samples), then a "single-letter" coding scheme achieves the optimum 
performance of ([T]), see e.g. lITTl . In this scheme, the transmitter consists of multiplication by a constant 
factor that adjusts the source to the power constraint P, so it is independent of the channel parameters. 

'it turns out, that for the purpose of analysis it is more convenient to use a colored-noise channel model rather than an ISI 
one; this is deferred to Section Ull 



Only the receiver needs to know the power of the noise in order to optimally estimate the source from 
the noisy channel output (by multiplying by the "Wiener coefficient"). 

For the case of colored sources and channels, however, such a simple solution is not available, as 
single-letter codes are only optimal in very special scenarios |10]. A particular case is when the channel 
bandwidth is not equal to the source bandwidth, but otherwise they are white (i.e., a white source is sent 
through an AWGN channel with some average number of channel uses per source sample). As it turns 
out, even if we consider more general linear transmission schemes, UJ, still ([T]) is not achievable in the 
general colored case. How far do we need to deviate from "analog" transmission in order to achieve 
optimal performance in the colored case? More importantly, can we still achieve full robustness? 

In this work we propose and investigate a semi-analog transmission scheme. This scheme achieves 
the optimum performance of ([U for any colored source and channel pair without explicit digital coding, 
hence we call it the Analog Matching (AM) scheme. Furthermore, for the matching bandwidth case 
(Be = Bs, but arbitrary source and channel spectra), we show that the Analog Matching transmitter is 
asymptotically robust in the high signal-to-noise ratio (SNR) regime, in the sense that it becomes invariant 
to the variance of the channel noise. Thus, in this regime, the perfect SNR-invariant matching property 
of white sources and channels llllll generalizes to the equal-BW colored case. 

Previous work on joint source/channel coding for the BW-mismatch/colored setting mostly consists 
of hybrid digital analog (HDA) solutions, which involve splitting the source or channel into frequency 
bands, or using a superposition of encoders (see |[25l . lITSl . ll24l . |[22l . IT91 and references therein), 
mostly for the cases of bandwidth expansion (Be > Bs) and bandwidth compression (Be < Bs) with 
white spectra. Most of these solutions, explicitly or implicitly, allocate different power and bandwidth 
resources to analog and digital source representations, thus they still employ full digital coding. Other 
works 111, |[30l treat bandwidth expansion by mapping each source sample to a sequence of channel 
inputs independently; by the scalar nature of these mappings, they do not aim at optimal performance. 

In contrast to HDA solutions, the AM scheme treats the source and channel in the time domain, 
using Unear prediction, thus it also has the potential of shorter delay. Furthermore, it does not involve 
any quantization of the source or digital channel code, but rather it applies modulo-lattice arithmetic to 
analog signals. This modulation allows to take advantage of side information - here based on prediction 
- while keeping the analog nature of transmission. 

Table U demonstrates the place of the Analog Matching scheme within information-theoretic time- 
domain schemes. For the separate colored Gaussian source and channel problems, digital coding schemes, 
based upon the combination of prediction and memoryless codebooks, are optimal: differential pulse code 



Problem 


Conventional prediction 


Side-information based solution 


Source coding 


DPCM compression 


WZ video coding 


Channel coding 


FFE-DFE receiver 


Dirty-paper coding = precoding 


Joint source-channel coding 


Does not exist 


Analog matching 



Table I: Information-Theoretic time-domain solutions to colored Gaussian source and channel problems. 



modulation (DPCM) in source coding (see fTT] for basic properties and fSFl for optimality), and feed- 
forward-equalizer / decision-feedback-equalizer (FFE-DFE) receiver in channel coding (see |3]). □ 

The optimality of DPCM hinges on prediction being performed using the reconstruction rather than the 
source itself. Bl Identical predictors, with equal outputs, are employed at the encoder and at the decoder. An 
alternative approach, advocated for low-complexity encoding, is "Wyner-Ziv video coding" (see e.g. f23\). 
In this approach, prediction is performed at the decoder only and is treated as decoder side-information 
|[32]| . In the context of the AM scheme, however, decoder-only prediction is not an option but a must: 
since no quantization is used, but rather the reconstruction error is generated by the channel, the encoder 
does not have access to the error and the side-information approach must be taken. 

In the channel counterpart, the FFE-DFE receiver cancels the effect of past channel inputs by filtering 
past decoder decisions (assumed to be equal to these inputs). In order to avoid error propagation, 
sometimes precoding |[27l . where the filter is moved to the encoder, is preferred; this can be seen as a 
form of dirty-paper coding |]4l , where the filter output plays the role of encoder side-information. Again, 
the AM scheme must use the "encoder side-information" variant: if no channel code is used, then the 
decoder cannot make digital decisions regarding past channel inputs, so virtually it has no access to these 
inputs. 

To summarize, the AM scheme uses source prediction at the decoder, and channel prediction at the 
encoder, and then treats the predictor outputs as Wyner-Ziv and dirty -paper side -information, respectively; 
see Figure [1] Digital solutions to these side-information problems rely on binning, which may also be 
materialized in a structured (lattice) way |[36l . AM treats these two side-information problems jointly 



In the high-rate limit it is easy to see the role of prediction: the rate-distortion function amounts to that of the white source 
innovations process, while the channel capacity is the additive white Gaussian noise channel capacity with the noise replaced 
by its innovations process only. We stick to this limit in the introduction; for general rates, see Section HIl 

^Extracting the innovations of the un-quantized source is sometimes called "D*PCM" and is known to be strictly inferior to 
DPCM; see QU, (35]. 
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Figure 1 : Workings of the AM scheme in the high-SNR limit. The source is assumed to have an auto- 
regressive (AR) model, mod A is the modulo-lattice operation. 

using modulo-lattice modulation (MLM) , an approach proposed recently for joint Wyner-Ziv and dirty- 
paper coding 111511 . However, combining these pieces turns out to be a non-trivial task. The interaction 
of filters with high-dimensional lattice codes raises technical difficulties which are solved in the sequel. 
The rest of the paper is organized as follows: We start in Section Ull by bringing preliminaries regarding 
sources and channels with memory, as well as modulo-lattice modulation and side-information problems. 
In Section JII] we prove the optimality of the Analog Matching scheme. In Section |IV] we analyze the 
scheme performance for unknown SNR, and prove its asymptotic robustness. Finally, Section Ivl discusses 
applications of AM, and is advantage relative to other approaches (e.g. HDA) in terms of delay. 



II. Formulation and Preliminaries 

In Section III-AI we formally present the problem. In the rest of the section we bring preliminaries 
necessary for the rest of the paper. In Sections III-BI to III-DI we present results connecting the Gaussian- 
quadratic rate-distortion function (RDF) and the Gaussian channel capacity to prediction, mostly following 
1351 . In sections [II-EI and HI-FI we discuss lattices and their application to joint source/channel coding 
with side information, mostly following |[T5l . 



A. Problem Formulation 

Figure |2] demonstrates the setting we consider in this paper. The source Sn is zero-mean stationary 
Gaussian, with spectrum Ss{e^'^'"^)- As for the channel, for the purpose of the analysis to follow we 
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Figure 2: Colored Gaussian joint source/channel setting. 



break away with the ISI model discussed in the introduction, and use a colored noise model: |^ 

In = Xn + Zn, (2) 

where Xn and Yn are the channel input and output, Z„ is zero-mean additive stationary Gaussian noise 
with spectrum Sz{e^'^'"^), assumed to be finite for all 2|/| < Bq and infinite otherwise. The channel 
input Xn needs to satisfy the power constraint Var{X„} < P, and the distortion of the reconstruction 
Sn is given hy D = Varj^n - Sn}. 

B. Spectral Decomposition and Prediction 

Let An be a zero-mean discrete-time stationary process, with power spectrum SAie^"^^-^)- The Paley- 
Wiener condition is given by |[28l : 



< oo 



(3) 



where here and in the sequel logarithms are taken to the natural base. It holds for example if the spectrum 
SA{e^'^'"-^) is bounded away from zero. Whenever the Paley -Wiener condition holds, the spectrum has a 
spectral decomposition: 



SAie 



J'^^f) 



Qiz)Q* 



1 



z=j2nf 



Pe Sa 



where Q{z) is a monic causal filter, and the entropy-power Pe (Sa) of the spectrum is defined by: 



Pe{SA)= Pel SAie 



J-^^f' 



exp 



log( SA{e''''n)df 



The optimal predictor of the process An from its infinite past is 

P{z) = l-Q-\z) , 



(4) 



(5) 



(6) 



*The transition between the two models is straightforward using a suitable front-end filter at the receiver (provided that the 
ISI filter is invertible). 



a filter with an impulse response satisfying p„ = for all n < 0. The prediction mean squared error 
(MSE) is equal to the entropy power of the process: 

Y^iiAM'T^} = PeiSA) ■ (V) 

The prediction error process can serve as a white innovations process for AR representation of the process. 
We define the prediction gain of a spectrum SA^e^^'^-^) as: 

where the gain equals one if and only if the spectrum is white, i.e. fixed over all frequencies |/| < |. A 
case of special interest, is where the process is band-limited such that SA{e^'^^^) = V|/| > -^ where 
S < 1. In that case, the Paley-Wiener condition Q does not hold and the prediction gain is infinite. We 
re-define, then, the prediction gain of a process band-limited to B as the gain of the process downsampled 



I.e., 



I 



iI:nSA{e^'-f)df 
r(5) = , ^ , ^^ . (9) 



exp 



^ J_Yog[SA{e^'-n)df 
We will use in the sequel prediction from a noisy version of a process: Suppose that C„ = An + Wn, 
with Wn additive white with power 9. Then it can be shown that the noisy prediction error has variance 
(see e.g. I.35in : 

Yar{An\CTj} = Pe{SA + e)-e . (10) 

Note that for any 6 > 0, the spectrum SA{e^'^'^-^) + obeys (O, so that the conditional variance is non-zero 
even if An is band-limited. In the case 6 = 0, (ITOl ) collapses to Q. 

C. Water-Filling Solutions and the Shannon Bounds 

The RDF for a Gaussian source with spectrum Ss{e-''^'^^) under an MSE distortion measure is given 
by: 

R{D) = - log ^; , ^I df , (11) 

'a similar definition can be made for more general cases, e.g., when the signal is band-limited to some band which does not 
start at zero frequency. 



where the distortion spectrum D{e^'^'^-f) is given by the reverse water-fiUing solution: D{e^'^'^-f) 
inm(9s,S{e^'^'^-f)j with the water level 9s set by the distortion level D: 

.1/2 



D= / D{e^'^^f)df . 

J-l/2 



'-1/2 

The Shannon lower bound (SLB) for the RDF of a source band-limited to Bs is given by: 

R{D)>^log^^RsLB{D) , (12) 

where SDR, the signal-to-distortion ratio, is defined as: 

3j,j,Ayar^ (13) 

and Ts = ^{Ss) is the source prediction gain dill. This bound is tight for a Gaussian source whenever the 



distortion level D is low enough such that D < i?5min|j|<^^ Ss{e^'^'"^), and consequently D{e^'^'^^) = 
9s = -§- for all |/| < Bs- Note that the bound reflects a coding rate gain of Bs/2log{rs) with respect 
to the RDF of a white Gaussian source. 

The capacity of the colored channel (HJ) where the noise Z„ has spectrum Sz{e-''^^^), bandlimited to 
Be, is given by: 

where the optimum channel input spectrum P{e^^'^^) is given by the water-filling solution: P{e^'^'^^) = 

maxf^C" — Szie^'^^-^),0] inside the band, with the water level 9c set by the power constraint P: 

rBc/2 
p= / P{e^'^^f)df . 

J-Bc/2 

The Shannon upper bound (SUB) for the channel capacity is given by: 

C<^log[rc-(l + SNR)]^C5[/B , (15) 

where SNR, the signal-to-noise ratio, is defined as: 

SNR = ^= „ ,^ ^ , (16) 

and Tc = ^{Sz) is the channel prediction gain ([8]l. The bound is tight for a Gaussian channel whenever 
the SNR is high enough such that P > Bcmax^j^^^^ Sz{e^'^''^) - N and consequently 5z(e^^''^) + 
p(gi27r/-j _ Q^ _ + ^ Note that the bound reflects a coding rate gain of Bc/2\og{Tc) with respect 
to the AWGN channel capacity. 



We now connect the capacity and RDF expressions. In terms of the SDR and SNR defined above, and 

denoting the inverse of the RDF by R~^{-), the optimal performance ([TJ becomes: 

SDRopt A yar{g„} 

i?-i(C(SNR)) ^ ^ 



Let the bandwidth ratio be 



P = ^ ■ ^''^ 



Combining (IT2l ) with dTSl ). we have the following asymptotically tight upper bound on the Shannon 
optimum performance. It shows that the prediction gains product r^rc gives the total SDR gain relative 
to the case where the source and channel spectra are white. 

Proposition 1: 

SDR°P^ 
(TTSNRV - ^ ^ ' 

n 

with equality if and only if the SLB and SUB both hold with equality, u Furthermore, if the source and 
the channel noise both satisfy the Paley-Wiener condition ([3]) inside their respective bandwidths, then 
when the SNR is taken to infinity by increasing the power constraint P while holding the noise spectrum 

fixed: 

SDR"^* 
lim -^Z7^ = ^src . (19) 



D. Predictive Presentation of the Gaussian RDF and Capacity 

Not only the SLB and SUB in (IT2l ) and (fTSl ) can be written in predictive forms, but also the rate- 
distortion function and channel capacity, in the Gaussian case. These predictive forms are given in terms 
of the forward-channel configurations depicted in Figure [3] 

For source coding, let Fi{e^^'^^) be any filter with amplitude response satisfying 

where D{e^'^'^f) is the distortion spectrum materializing the water-filling solution (ITTT ). We call Fi{e^'^'^-f) 
and F2(e^27r/) ^ i?*(ei2T/^ (^e pre- and post-filters for the source S 041 . 

*The SLB and SUB never strictly hold with equality if S's(e^^'^^) is not bounded away from zero, or Sz{e.^^^^) is not 
everywhere finite. However, they do hold asymptotically in the high-SNR limit, if these spectra satisfy the Paley-Wiener condition 
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(a) RDF-achieving configuration using filters and AWGN channel. 
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(b) Capacity-achieving configuration using filters and white Gaussian 
input. 



Figure 3: Forward-channel configurations for tlie RDF and capacity. 



As a consequence of (ITOl ). the pre/post filtered AWGN depicted in Figure l3al satisfies |[35l : 



(21) 



where Var{Z„} = 9s- Note that in the limit of low distortion the filters vanish, prediction from C/„ is 
equivalent to prediction from Vn, and we go back to (fT2]) . Defining the source Wiener coefficient 



(I2TI ) implies that 



as = l- exp {-2R{D)) 
Var{t/„|y"^n 



as 
I -as 



0s 



(22) 



(23) 



(24) 



For channel coding, let Gi{e^'^'^f) be any filter with amplitude response satisfying 

\Gi{e-' ' )\ - — , 

where P{e^'^'^f) and 9c are the channel input spectrum and water level materializing the water-filling 
solution (IT4l ). Gi{e^'^'^^) is usually referred to as the channel shaping filter, but motivated by the the 
similarity with the solution to the source problem we call it a channel pre-filter. At the channel output 
we place G2{e^'^^f) = G\{e^'^'^f), known as a matched filter, which we call a channel post-filter. 

In the pre/post filtered colored-noise channel depicted in Figure [3bl let the input Xn be white and 
define the (non-Gauusian, non-additive) noise Z„ = Yn — Xn- Then the channel satisfies (see 111, ||35l ): 



C = - log , ^^ — ^ 



(25) 



10 



where Var{X„} = 9c- Note that in the Umit of low noise the filters vanish, prediction from Z„ is 
equivalent to prediction from Zn, and we go back to (fTSl ). Defining the channel Wiener coefficient 

ac = l- exp (-2C) , (26) 

(l25l) implies that 



Var{Z„|Z!!-i} = ^^^^c . (27) 

ac 

Combining (l22l) with (l26l ). we note that in a joint source-channel setting where the optimum perfor- 
mance ([T]) is achieved, 

as = ac = a . (28) 



A connection between the water-filling parameters and conditional variances can be derived using (1231) 
and ( |27l ). 

The predictive presentations (|2T]) and ( |25] ) translate the process mutual information rates 7(S'„;S'„) 
and /(X„;y„) to the conditional mutual informations /([/„; T4 1 K!*oo^) and /(X„;X„ + Z„|Z"J^^), 
respectively. This is highly attractive as the basis for coding schemes, since it allows to use the combination 
of predictors and generic optimal codebooks for white sources and channels, regardless of the actual 
spectra, without compromising optimality. See e.g. llT2l . |[35l . In the source case, (|2T| | establishes the 
optimality of a DPCM-like scheme, where the prediction error of f7„ from the past samples of Vn is 
being quantized and the quantizer is equivalent to an AWGN. For channel coding, ( |25] ) implies a noise- 
prediction receiver, which can be shown to be equivalent to the better known MMSE FFE-DFE solution 

m. 

E. Good Lattices for Quantization and Channel Coding 

Let A be a K-dimensional lattice, defined by the generator matrix G S M^^^. The lattice includes 

all points {1 = G • i : i G Z^} where Z = {0, ±1, ±2, . . .}. The nearest neighbor quantizer associated 

with A is defined by 

Q(x) = argmin llx — 111 , 
iga 

where ties are broken in a systematic way. Let the basic Voronoi cell of A be 

Vo = {x : Q(x) = 0} . 

The second moment of a lattice is given by the variance of a uniform distribution over the basic Voronoi 

cell, per dimension: 

„ 1 L llxlpfix 

11 



The modulo-lattice operation is defined by: 

X mod A = X — (3(x) . 

The following is the key condition that has to be verified in the analysis to follow. 

Definition 1: (Correct decoding) We say that correct decoding of a vector x by a lattice A occurs, 
whenever 

X mod A = X , 

i.e., X G Vq. 

For a dither vector d which is independent of x and uniformly distributed over the basic Voronoi cell 

Vo, [x + d] mod A is uniformly distributed over Vo as well, and is independent of x ||33l . 

We assume the use of lattices which are simultaneously good for source coding (MSB quantization) 
and for AWGN channel coding Q. Roughly speaking, a sequence of K-dimensional lattices is good 
for MSE quantization if the second moment of these lattices tends to that of a uniform distribution over 
a ball of the same volume, as K grows. A sequence of lattices is good for AWGN channel coding if 
the probability of correct decoding ([T]) of a Gaussian i.i.d. vector with element variance smaller than 
the square radius of a ball having the same volume as the lattice basic cell, approaches zero for large 
K. There exists a sequence of lattices satisfying both properties simultaneously, thus for these lattices, 
correct decoding holds with high probability for Gaussian i.i.d. vectors with element variance smaller than 
cr^(A), for large enough K. This property also holds when the Gaussian vector is replaced by a linear 
combination of Gaussian and "self noise" (uniformly distributed over the lattice basic cell) components. 
The following formally states the property used in the sequel. 

Definition 2: (Combination noise) Let Zi,...,Zi be mutually-i.i.d. vectors, independent of Zq, 
uniformly distributed over the basic cell of A, and let Zq be a Gaussian i.i.d. vector with element variance 
cr^(A). Then for any real coefficients, Yli=o '^I'^L is a combination noise with composition ao? • • • > oil- 

Proposition 2: (Existence of good lattices) Let {A^} denote a sequence of i^-dimensional lattices 
with basic cells {Vk} of fixed second moment a'^. Let {Z^'} be a corresponding sequence of combination 
noise vectors, with fixed composition satisfying: 

1=1 

Then there exists a sequence {Ax} such that: 

limsupPrjZi^ mod Ak t^ Z^} = . 

12 
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Figure 4: The Wyner-Ziv / dirty-paper coding problem. 
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Figure 5: MLM Wyner-Ziv / dirty -paper coding. 



This is similar to [15, Poposition 1], but with the single "self-noise" component replaced by a 
combination. It therefore requires a proof, included in the appendix. 
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F. Coding for the Joint WZ/DPC Problem using Modulo-Lattice Modulation 

The lattices discussed above can be used for achieving the optimum performance in the joint source/channel 
Gaussian Wyner-Ziv/dirty-paper coding, [] depicted in Figure ID In that problem, the source is the sum of 
an unknown i.i.d. Gaussian component Qn and an arbitrary component J„ known at the decoder, while 
the channel noise is the sum of an unknown i.i.d. Gaussian component Zn and an arbitrary component /„ 
known at the encoder. In ifTSl the MLM scheme of Figure [5a] is shown to achieve the optimal performance 
([1]) for suitable a and /3. This is done showing asymptotic equivalence with high probability (for good 
lattices) to the real-additive channel of Figure [Sb] The output-power constraint P in that last channel 
reflects the element variance condition in order to ensure correct decoding of the vector /3Qn + Zeg„ 
with high probability, according to Proposition [2] When this holds, the dithered modulo-lattice operation 
at the encoder and the decoder perfectly cancel each other. This way, the MLM scheme asymptotically 
translates the SI problem to the simple problem of transmitting the unknown source component Qn over 
an AWGN, where the known source component J„ and the channel interference /„ are not present. 

III. The am Scheme 

In this section we prove the optimality of the Analog Matching scheme, depicted in Figure |6l in the limit 
of high lattice dimension. We assume for now that we have K mutually-independent identically-distributed 
source-channel pairs in parallel!^ which allows a ii'-dimensional dithered modulo-lattice operation across 
these pairs. Other operations are done independently in parallel. To simplify notation we omit the index 
k of the source/channel pair (k = 1,2, . . . , K), and use scalar notation meaning any of the K pairs; we 
denote by bold letters A'-dimensional vectors, for the modulo-lattice operation. Subscripts denote time 
instants. Under this notation, the AM encoder is given by: 

Un = fin * ^n 

X„ = [/3U„ - In + Dn] mod A 

oo 

-'n ^ ~ / ^ PCm^n—m 
m=l 

^n = gin * ^n , (30) 

^An alternative form of this scheme may be obtained by replacing the lattice with a random code and using mutual information 
considerations; see 1311 . 

^We will discuss in the sequel how this leads to optimality for a single source and a single channel. 
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Figure 6: The Analog Matching scheme. 



while the decoder is given by: 



Yn -- 


= 92n * Yn 






oo 


Y'n -- 


= ^n-Yl PCmXn-m 




ni=l 


v„ = 


- "5 I n ~ pJn -L'n 


mod A + J 




oo 


Jn - 


= Yl PSk^n-k 




m=l 


Sn -- 


= f2n*Vn , 





(31) 



where * denotes convolution. For each filter of frequency response H{e^'^'^f), the corresponding impulse 
response is denoted by small letters /i„. Each of the K parallel channels is given by the colored noise 
model ©. 

The filters used in the scheme are determined by the optimal solutions presented in Sections III-CI 
and III-DI The channel capacity, and corresponding water-level 6c, are given by (IT4l ). This determines, 
through the optimality condition ([T), the distortion level D. Using that D, the RDF, and corresponding 
water-level ^5, are given by (ITTI) . The filters Fi{e^'^'^^) and F2{e^'^'^^) are then chosen according to dlOl ). 



15 



and Giie^'^^'f) and G2{e^^''f) according to dH. We also use a of dHJl. Finally, ^^(e^^Tr/^ ^^^ Pde^'^'^f) 
are the optimal predictors (O of the spectra 

5v(e^-2-/) ^ |Fi(e^-2-/)|255(e^-2-/) + 1-^0^ (32) 



and 



2 



respectively, where we take \Gi{e^'^''f)\'^Sz{e^'^''^) = wherever \Gi{e^'^''f)\ = even if Sz{e^'^'^^) is 
infinite. 

The analysis we apply to the scheme shows that at each time instant it is equivalent to a joint 
source/channel side-information (SI) scheme, and then applies the Modulo-Lattice Modulation (MLM) 
approach presented in Section HI-FI The key to the proof is showing that with high probability the correct 
decoding event of Definition [J holds, thus the modulo-lattice operation at the decoder exactly cancels the 
corresponding operation at the encoder. As the distribution of the signal fed to the decoder modulo-lattice 
operation depends upon the past decisions through the filters memory, the analysis has a recursive nature: 
we show, that if the scheme is in a "correct state" at time instant n, it will stay in that state at instant 
n+1 with high probability, resulting in optimal distortion. Formally, for the decoder modulo-lattice input: 

T„ = /3(U„ - Jn) + Zeg„ , (34) 

we define the desired state as follows. 

Definition 3: We say that the Analog Matching scheme is correctly initialized at time instance n, if 
all signals at all times n — l,n — 2, . . . take values according to the assumption that correct decoding 
(see Definition [D held w.r.t. T„_i, T„_2, .... 

Using this, we make the following optimality claim. 

Theorem 1: (Asymptotic optimality of the Analog Matching scheme) Let D{N, K) be the achievable 
expected distortion of the AM scheme operating on input blocks of time duration A^ with lattice dimension 
K. Then there exists a sequence N{K) such that 

lim D{N{K),K) = D°P* , 

A'— ^-oo 

where D°p^ was defined in ([T]), provided that at the start of transmission the scheme is correctly initialized. 
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In Section IIII-AI we gain some insight into the workings of the scheme by considering it in the 
uqual-BW high-SNR regime, while in Section ITlI-B I we cosider the important special cases of bandwidth 
expansion and compression. Section IIII-CI contains the proof of Theorem \T\ and then in Section IIII-DI 
we discuss how one can implement a scheme based on the theorem. 



A. The Scheme in the Equal-BW High-SNR Regime 

In the equal-BW case (Bs = Be) and in the limit of high resolution (D"p* <C Pe{Ss)), the scheme can 
be simplified significantly. In this limit, the filters approach zero-forcing ones: both source and channel 
pre- and post-filters collapse to unit all-pass filters, while the source and channel predictors become just 
the optimal predictors of the spectra Ss{e^^'^-^) and Sz{e^'^'"^), respectively. The receiver filter 1 — Pc{z) 
is then a whitening filter for the noise, and the channel from X„ to Y^ is equivalent to a white-noise 
inter-symbol interference (ISI) channel: 



^n = ^n — 2^ PCm^n~m + Wn , 



(35) 



m=l 



where Wn is AWGN of variance Pe{Sz)- Under these conditions, the source can always be modeled as 
an auto-regressive (AR) process: 



^n 



oo 

/ ^ PSm'-'n—m 
m=l 



PSm^n—m ~r t^n j 



(36) 



where Qn is the white innovations process, of power P(,{Ss)- 

The resulting scheme is depicted in Figure |7J It is evident, that the channel predictor cancels all of 
the ISI, while the source predictor removes the source memory, so that effectively the scheme transmits 
the source innovation through an AWGN channel. The gains of source and channel prediction are Ts = 
PeiSs)/yar{Sn} and Tc = Pe(5'z)/ Var{Z„}, respectively (recall dHJl). In light of ([Hi, the product 
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of the two is indeed the required gain over memoryless transmission. In fact, if we assume that the 
modulo-lattice operations have no effect, then the entire scheme is equivalent to the AWGN channel: 

A _ c. , ^n 

Letting /3^ = P/VarjQ^}, we than have that: 

SDR = —- — = = i si c^NK 

Ya.r{Wn} P 

which is optimal at the limit, recall ( fT9l ). In order to satisfy the power constraint, the lattice second 

moment must be P, thus the gain /3 amplifies the source innovations to a power equal to the lattice 

second moment; as we will prove in the sequel, this choice of /3 indeed guarantees correct decoding, on 

account of Proposition |2] 

B. The BW Mismatch Case 

At this point, we present the special cases of bandwidth expansion and bandwidth compression, and 
see how the analog matching scheme specializes to these cases. In these cases the source and the channel 
are both white, but with different bandwidth (BW). The source and channel prediction gains are both 
one, and the optimum condition (fTTl ) becomes: 

SDR^P* = (1 + SNR)^ , (37) 

where the bandwidth ratio p was defined in ( fTSl ). 

For bandwidth expansion {p > 1), we choose to work with a sampling rate corresponding with the 
channel bandwidth, thus in our discrete-time model the channel is white, but the source is band-limited to 
a frequency of 2^. As a result, the channel predictor Pc{z) vanishes and the channel post-filters become 
the scalar Wiener factor a. The source water-filling solution allocates all the distortion to the in-band 
frequencies, thus we have 9s = pD and the source pre- and post-filters become both ideal low-pass filters 
of width J- and height 

i-^5^sFt = yi- (i+sNR)^ ■ ^^^^ 

As the source is band-limited, the source predictor is non-trivial and depends on the distortion level. The 
resulting prediction error of Un has variance 



(1 + SNR)''"^ ' 
and the resulting distortion achieves the optimum (l37l) . 
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For bandwidth compression (p < 1), the sampUng rate reflects the source bandwidth, thus the source 
is white but the channel is band-Umited to a frequency of Be = |. In this case the source predictor 
becomes redundant, and the pre- and post-filters become a constant factor equal to ( [38] ). The channel 
pre- and post-filters are ideal low-pass filter of width | and unit height. The channel predictor is the 
SNR-dependent DFE. Again this results in achieving the optimum distortion ( [37] ). It is interesting to note, 
that in this case the outband part of the channel error Z„ is entirely ISI (a filtered version of the channel 
inputs), while the inband part is composed of both channel noise and ISI, and tends to be all channel 
noise at high SNR. 

C. Proof of Theorem |7] 

We start the optimality proof by showing that the Analog Matching scheme is equivalent at each time 
instant to a WZ/DPC scheme, as in Section HI-FI Specifically, the equivalent scheme is shown in Figure 
|8al which bears close resemblance to Figure |5al The equivalence is immediate using the definitions of 
In (l30l) and J„ (IBTT i. since they are constructed in the encoder and the decoder using past values of X„ 
and Vn, respectively, thus at any fixed time instant they can be seen as side information. It remains to 
show that indeed the unknown noise component is white, and evaluate its variance. 

Lemma 1: (Equivalent side-information scheme) Assume that VarX„ = 6c, then 

a 



^ n — ^n 



is a white process, independent of all C/„, with variance 

1 — o- 

Var{Z;} = Qc 

a 



The proof of this lemma appears in the appendix. Now we note, that if the modulo-lattice operations 
in the equivalent scheme of Figure [8a] can be dropped, this will result in a scalar additive white-noise 
channel (see Figure [8]): 

Vn = Un + ^ , (39) 

where 

Zeg^ = aZ'^ - (1 - a)Xr, (40) 

is a white additive noise term of variance 

Var{Ze,„} = aVar{Z;} = (l-a)ec ■ (41) 
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Figure 8: Equivalent channels for the Analog Matching scheme. 



F„ 



Together with the source pre/post filters F\(e?'^'^^^ and F2{e^^'^^), we can have the forward test channel 
of Figure [3al Furthermore, if /S^ equals 



„2 A . .Oc 

P, = il-a) — 



(42) 



then the additive noise variance is 9s, resulting in optimal performance. The relevant condition is that 
correct decoding (recall Definition [D holds for T„ (l34l) . Using the concept of correct initialization 
(Definition O, we first give a recursive claim. 

Lemma 2: (Steady-state behavior of the Analog Matching scheme) Assume that the Analog Match- 
ing scheme is applied, using a lattice A = Ak of dimension K which is taken from a sequence of lattices 
of second moment 9c which are simultaneously good for source and channel coding in the sense of 
Proposition [2] Then the probability that correct decoding does not hold in the present instance can be 
bounded by pe{K), where 

lim peiK) = , 



given that the scheme is correctly initialized and that /3 > /3o (|42 
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Now we translate the conditional result above (again proven in the appendix), to the optimality claim 
for blocks. 

Proof of Theorem [ij We choose N{K) to be some sequence such that 



but at the same time 



lim N{K) = oo , 

7f— >-oo 



lim N{K)pe{K) = , 



where Pe{K) was defined in Lemma |2] Let D'^°^^'^'^^ and j)''-^'^"^^'^'^^ be the expected distortion given that 
the scheme remains correctly initialized at the end of transmission or does not, respectively. By the union 
bound we have that: 

D{N{K), K) < D''''"-''^\N{K),K) + N{K)pe{K)D^''^''"^''\N{K), K) . 

Since we assumed that N{K)pe{K) vanishes in the limit of infinite K, so does the second term; see 
|[T5l Appendix II-B]. We thus have that 

lim D{N{K),K)= lim D'"'"'^^\N{K),K) 



and we can assume that ( 1391 ) holds throughout the block. This results in the forward channel of Figure 
[3al up to two issues. First, the channel is stationary while transmission has finite duration, and second 
the additive noise variance is larger than 6s since (3 > I3q. The first may be solved by forcing CZ„ and 
Vn to be zero outside the transmission block, resulting in an excess distortion term; however this finite 
term vanishes when averaging over large N{K). The second implies that D°p* + e may still be achieved 
for any e > 0, and the result follows by a standard arguments, replacing e by a sequence e{K) — > 

D. From the Idealized Scheme to Implementation 

We now discuss how the scheme can be implemented with finite filters, how the correct initialization 
assumption may be dropped, and how the scheme may be used for a single source/channel pair. 

1 . Filter length. If we constrain the filters to have finite length, we may not be able to implement the 
optimum filters. However, it is possible to show, that the effect on both the correct decoding condition 
and the final distortion can be made as small as desired, since the additional signal errors due to the 
filters truncation can be all made to have arbitrarily small variance by taking long enough filters. In the 
sequel we assume the filters all have length L. 

2. Initialization. After taking finite-length filters, we note that correct initialization now only involves a 
finite history of the scheme. Consequently, we can create this state by adding a finite number of channel 
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uses. Now we may create a valid state for the channel predictor Pc{e^'^'^^) by transmitting L values 
Xn = 0; see |[T2l . For the source predictor the situation is more involved, since in absence of past values 
of Vn, the decoder cannot reduce the source power to the innovations power, and correct decoding may 
not hold. This can be solved by de-activating the predictor for the first L values of C/„, and transmitting 
them with lower /? such that (l54l ) holds without subtracting J„. Now in order to achieve the desired 
estimation error for these first values of Vn, one simply repeats the same values of C/„ a number of times 
according to the (finite) ratio of /3's. If the block length N is long enough relative to L, the number of 
excess channel uses becomes insignificant. 

3. Single source/channel pair. A pair of interleaver/de-interleaver can serve to emulate K parallel 
sources, as done in |[T2l for an FFE-DFE receiver, and extended to lattice operations in ||33. Interestingly, 
while a separation-based scheme which employs time-domain processing for both source and channel 
parts requires two separate interleavers, one suffices for the AM scheme. Together with the initialization 
process, we have the following algorithm. 

encoder: 

1. Write Un row-wise into an interleaving table. 

2. Before each row, add source initialization samples. 

3. Build a table for Xn, starting by zero columns for channel initialization. Then add more columns 
using column-wise modulo-lattice operations on the table of Un, using row-wise past values of Xn as 
inputs to the channel predictor. 

4. Feed the Xn table to the channel pre-filter row-wise. 
decoder: 

1. Write y^ row-wise. 

2. Discard the first columns, corresponding to channel initialization. 

3. Build a table for Vn, starting by using the source initialization data. Then add more columns using 
column-wise modulo-lattice operations on the table of y^, using row-wise past values of Vn as inputs to 
the source predictor. 

4. Feed the Vn table to the source post-filter row-wise. 

IV. Unknown SNR 

So far we have assumed in our analysis that both the encoder and decoder know the source and channel 
statistics. In many practical communications scenarios, however, the encoder does not know the channel, 
or equivalently, it needs to send the same message to different users having different channels. Sometimes 
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it is assumed that the channel filter HQ{e^'^'^f) is given, but the noise level N is only known to satisfy 
N < Nq for some given Nq. For this special case, and specifically the broadcast bandwidth expansion 
and compression problems, see |[25l . ifTSl . |[24l . | [22l . 

Throughout this section, we demonstrate that the key factor in asymptotic behavior for high SNR is 
the bandwidth ratio p (fTSl ). We start in Section IIV-AI by proving a basic lemma regarding achievable 
performance when the encoder is not optimal for the actual channel. In the rest of the section we utilize 
this result: in Section IIV-BI we show asymptotic optimality for unknown SNR in the case p = 1, then 
in Section IIV-CI we show achievable performance for the special cases of (white) BW expansion and 
compression, and finally in Section IIV-DI we discuss general spectra in the high-SNR limit. 

A. Basic Lemma for Unknown SNR 

We prove a result which is valid for the transmission of a colored source over a degraded colored 
Gaussian broadcast channel: We assume that the channel is given by Q, where Be is known but the noise 
spectrum Sz{e^'^'"^) is unknown, except that it is bounded from above by some spectrum Szo{e^'^^^) 
everywhere. We then use an Analog Matching encoder optimal for Szo{e^'^'^-^), as in Theorem [U but 
optimize the decoder for the actual noise spectrum. Correct decoding under S zoie-''^'^ ^ ) ensures correct 
decoding under Sz{e^'^'"^), thus the problem reduces to a linear estimation problem, as will be evident 
in the proof. 

For this worst channel Szo{e^'^'^^) and for optimal distortion ( [TT] ). we find the water-filling solutions 
(ITTI).(fT4l). resulting in the source and channel water levels 6$ and 9c respectively, and in a source-channel 
passband Tq, which is the intersection of the inband frequencies of the source and channel water-fiUing 
solutions: 

^s = {/ : 5s(e^'"0 > %} , 

To = -^snJ-c . (43) 

Under this notation we have the following lemma, proven in the appendix. It shows that the resulting 
distortion spectrum is that of a linear scheme which transmits the source into a channel with noise 
spectrum P/^{e^'^'^f), where 



$(e^'2^/) 



Szoie- 



.j'2nf] 



Szie^^^n 



Oc 



9s 



depends on both the design noise spectrum and the actual one. 
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Lemma 3: For any noise spectrum Szo{e^'^^^), there exists an encoder, such that for any equivalent 
noise spectrum 

Sz{e'^^f)<Sz»{e^^^f)^ f eFc , (44) 

a suitable decoder can arbitrarily approach: 

D = j\D{e^^^f)df , 

2 

where the distortion spectrum D{e^'^'^^) satisfies: 

f Ss{e^^-n if f a -F 1 

[ mm( Ssie^^''^),Os), otherwise J 

Remarks: 

1. Outside the source-channel passband Tq, there is no gain when the noise spectrum density is lower 
than expected. Inside Tq, the distortion spectrum is strictly monotonously decreasing in Sz{e.^'^^f), but 
the dependence is never stronger than inversely proportional. It follows, that the overall SDR is at most 
linear with the SNR. This is to be expected, since all the gain comes from linear estimation. 

2. In the unmatched case modulation may change performance. That is, swapping source frequency 
bands before the analog matching encoder will change Tq and <I>(e-'^'^'^), resulting in different performance 
as Sz{e-^'^'^^) varies. It can be shown that the best robustness is achieved when Ss{e^'^'^f) is monotonously 
decreasing in Sz{e^'^'"^). 

3. The degraded channel condition (l44l ) is not necessary. A tighter condition for correct decoding to 
hold can be stated in terms of Ss{e^'^''^), Szoie^"^""^) and Sz{e^'^'^f): the integral over Seq{e^'^''^), defined 
in the appendix ( [55] ). must be at most as it is for the spectrum 5*0 (e-'^'^-^). 

B. Asymptotic Optimality for equal BW 

We prove asymptotic optimality in the sense that, if in an ISI channel (recall ([35l)). the ISI filter is 
known but the SNR is only known to be above some SNRq, then a single encoder can simultaneously 
approach optimality for any such SNR, in the limit of high SNRq. 

Theorem 2: (High-SNR robustness) Let the source and channel have BW Bs = Be = 1, and let 
the equivalent ISI model of the channel (l35l) have fixed filter coefficients (but unknown innovations 
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power Var{Wn}). Then, there exists an SNR-independent sequence of encoders indexed by their lattice 
dimension K, each achieving SDRii:(SNR), such that for any 5 > 0: 

lim SDRx(SNR) > (1 - (5)SDR°p*(SNR) 

for sufficiently large (but finite) SNR, i.e., for all SNR > SNRo((5). 

Proof: The limit of a sequence of encoders is required, since any fixed finite-dimensional encoder 
has a gap from SDR°^* that would limit performance as SNR -^ oo. At the limit, however, we may 
assume an ideal scheme. In terms of the colored noise channel Q, the unknown noise variance in the 
theorem conditions is equivalent to having noise spectrum 

where SNR > SNRq = SNRo(5). We apply Lemma [3 with an encoder designed for Szoie^^""^)- If the 

source spectrum is bounded away from zero and the Szoie-^"^^^) is bounded from above, we can always 

take SNRq high enough such that the source-channel passband Jq includes all frequencies, and then we 

have for all SNR > SNRq: 

.„ . 1 SNRn 

Os 



D{e^'-f)<J- ^^^'' 



1-6 SNR 
resulting in 

SNR SNR 

SDR > (1 - 5)^^SDRo = (1 - 5)^^r5rc;(l + SNRo) = (1 - 5)SDR°p* 

where the equalities are due to Proposition \T\ If the spectra are not bounded, then we artificially set the 
pre-filters to be 1 outside their respective bands (and apply an additional gain in order to comply with 
the power constraint). This inflicts an arbitrarily small SDR loss at SNRq, but retains SDR oc SNR, thus 
the gap from optimality can be kept arbitrarily small. ■ 

Alternatively, we could prove this result using a the zero-forcing scheme of Figure |7] In fact, using 
such a scheme, an even stronger result can be proven: not only can the encoder be SNR-independent, 
but so can the decoder. 

C. BW Expansion and Compression 

We go back now to the cases of bandwidth expansion and compression discussed at the end of Section 
Inll In these cases, we can no longer have a single Analog Matching encoder which is universal for 
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different SNRs, even in the high SNR Umit. For bandwidth expansion (p > 1), the reason is that the 
source is perfectly predictable, thus at the limit of high SNR we have that 

Var{t/„|K-i, Vn-2, . . .} ^ Var{C/„|[/,_i, C/„_2, . . .} = , 

thus the optimum /3 goes to infinity. Any /3 value chosen to ensure correct decoding at some finite SNR, 
will impose unbounded loss as the SNR further grows. For bandwidth compression, the reason is that 
using any channel predictor suitable for some finite SNR, we have in the equivalent noise Zn = Yn — Xn 
some component which depends on the channel input (set by the dither). As the SNR further grows, this 
component does not decrease, inflicting again unbounded loss. 

By straightforward substitution in Lemma |3j we arrive at the following. 

Corollary 1: Assume white source and AWGN channel where we are allowed p channel uses per 
source sample. Then using an optimum Analog Matching encoder for signal to noise ratio SNRq and a 
suitable (SNR-dependent) decoder, it is possible to approach for any SNR > SNRq: 

1 l-mm{l,p) mm{l,p) 

~ P + -I , ^ /OTVTT^ OTVTT^ X ' (46) 



SDR fi + SNRo) l + '^p(SNR,SNRo) 



where 



$p(SNR,SNRo 



A 1 + SNR 
~ 1 + SNRo 



:i + SNRo)" - 1 



(47) 



Note that the choice of filters in the SNR-dependent decoder remains simple in this case: For p > 1 
the channel post-filter is flat while the source post-filter is an ideal low-pass filter, while for p < 1 it is 
vice versa. The only parameters which change with SNR, are the scalar filter gains. 

Comparison of performance: In comparison, the performance reported by different methods in [181 . 
ll24l for these cases has, in terms of (l46l ): 



$p(SNR, SNRo) = (1 + SNR) • (1 + SNRo)''"^ - 1 (48) 



while f24l also proves an outer bound for BW expansion (p > 1) on any scheme which is optimal at 

some SNR: 

SNR r 1 

(49) 



SNR 

$p(SNR, SNRo) = ^^f^ (1 + SNRo)" - 1 



In both BW expansion and compression, the Analog Matching scheme does not perform as good as 
the previously reported schemes, although the difference vanishes for high SNR. The basic drawback of 
analog matching compared to methods developed specifically for these special cases seems to be, that 
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Figure 9: Unknown SNR performance: BW expansion and compression. The best known achievable 
performance, brought for comparison, is due to II18I . |[24ll . 



these methods apply different "zooming" to different source or channel frequency bands, analog matching 
uses the same "zooming factor" /3 for all bands. Enhancements to the scheme, such as the combination 
of analog matching with pure analog transmission, may improve these results. Figure [TV-CI demonstrates 
these results, for systems which are optimal at different SNR levels. 

At high SNR, the performance of all these methods and of the outer bound converge to: 



1 1 — niin(p, 1) 



SDR 



SNRg 



+ 



min(/9, 1) 



SNR • SNR; 



p-i 



(50) 



Thus the Analog Matching scheme, as well as the schemes of 11181 . 11241 . are all asymptotically optimal 
for high SNR among the schemes which achieve SDR°p* at some SNR. 



D. Asymptotic Behavior with EW Change 

Finally we turn back to the general case of non-white spectra with any p, and examine it in the high- 
SNR regime. As in Section IIV-BI we assume that the channel ISI filter is known, corresponding with an 
equivalent noise spectrum Szie?'^'"^) known up to a scalar factor. 

In the high-SNR limit. Lemma |3] implies: 



1 



SDR 



1 — min(/0, 1) 
SNRg 



+ 



min(p, 1) 



SNR • SNR: 



p-i 



TcFs 



(51) 



27 



Comparing with (pOl l. we see that the color of the source and of the noise determines a constant factor 
by which the SDR is multipUed, but the dependence upon the SNR remains similar to the white BW 
expansion/compression case. The following definition formahzes this behavior (see lfT6l ). 

Definition 4: The distortion slope of a continuum of SNR-dependent schemes is : 

SNR^oo log SNR 
where SDR is the signal to distortion attained at signal to noise ratio SNR, where the limit is taken for 
a fixed channel filter with noise variance approaching 0. 

We use the notation A = A(p) in order to emphasize the dependance of the asymptotic slope upon the 
bandwidth expansion factor. The following follows directly from Proposition [T] 

Proposition 3: For any source and channel spectra with BW ratio p, and for a continuum of schemes 
achieving the OPTA performance (ITT] ). 

X{p) = p . 



As for an analog matching scheme which is optimal for a single SNR, (I5TI ) implies: 

Corollary 2: For any source and channel spectra and for a single analog-matching encoder, 

f 1, if p > 1 
I 0, otherwise 
is achievable. 

This asymptotic slope agrees with the outer bound of f2M for the (white) bandwidth expansion problem. 
For the bandwidth compression problem, no outer bound is known, but we are not aware of any proposed 
scheme with a non-zero asymptotic slope. We believe this to be true for all spectra: 

Conjucture 1: For any source and channel spectra of BW ratio p, no single encoder which satisfies 
dTTl) at some SNRq can have a better slope than that of Corollary |2] 

By this conjecture, the analog matching encoder is asymptotically optimal among all encoders ideally 
matched to one SNR. It should be noted, that schemes which do not satisfy optimality at one SNR can 
in fact approach the ideal slope X{p) = p. See e.g. approaches for integer p such as bit interleaving |[26]| . 
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V. Conclusion: Implementation and Applications 

We presented the Analog Matching scheme, which optimally transmits a Gaussian source of any 
spectrum over a Gaussian channel of any spectrum, without resorting to any data-bearing code. We 
showed the advantage of such a scheme over a separation-based solution, in the sense of robustness for 
unknown channel SNR. 

The analysis we provided was asymptotic, in the sense that a high-dimensional lattice is needed. 
However, unlike digital transmission (and hybrid digital-analog schemes) where reduction of the code 
block length has a severe impact on performance, the semi-analog approach offers a potential advantage 
in terms of block-length. An asymptotic figure of merit where we expect this advantage to be revealed, is 
the excess-distortion exponent. Furthermore, the modulo-lattice framework allows in practice reduction 
to low-dimensional, even scalar lattices, with bounded loss. 

One approach for scalar implementation of the Analog Matching scheme, uses companding ifTTl . In 
this approach, the scalar zooming factor /? is replaced by a non-linear function which compresses the 
unbounded Gaussian source into a finite range, an operation which is reverted at the decoder. There is 
a problem here, since the entity which needs to be compressed is actually the innovations process Qn, 
unknown at the encoder since it depends on the channel noise. This can be solved by compressing Qn, 
the innovations of the source itself; The effect of this "companding encoder-decoder mismatch" vanishes 
in the high-SNR limit. An altogether different approach, is to avoid instantaneous decoding of the lattice; 
Instead, the decoder may at each instance calculate the source prediction using several hypothesis in 
parallel. The ambiguity will be solved in future instances, possibly by a trellis-like algorithm. 

In terms of delay, the AM scheme has an additional advantage over previously suggested HDA schemes. 
It is well known that time-domain approaches have a delay advantage over frequency-domain one, in 
both source and channel coding. A fully-causal DPCM, for example, can approach the RDF while only 
using causal filters, on the high-resolution limit. A sub-band coding scheme, in contrast, would have to 
use a delay-consuming DFT block; see e.g. [13|. 

Finally, we remark that the AM scheme has further applications. It possesses the basic property, that 
it converts any colored channel to an equivalent additive white noise channel of the same capacity 
as the original channel, but of the source bandwidth. In the limit of high-SNR, this equivalent noise 
becomes Gaussian and independent of any encoder signal. This property is plausible in multi-user source, 
channel and joint source/channel problems, in the presence of bandwidth mismatch. Applications include 
computation over MACs 1(211 . multi-sensor detection [20] and transmission over the parallel relay network 
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Appendix 
A. Proof of Proposition |2] 

By ||8] (200)], for each of the components Zf. 

-log^<.(A^) (53) 

n fz[(z) 

where /(•) denotes a probabiUty density function (pdf), Z[ is AWGN with the same variance as Zi, and 
e(A/^) — )• as ii' — ;■ oo for a sequence of lattices which is Rogers-good (i.e. lattices for which volume 
of the covering sphere approaches that of the Voronoi cell). Now assume without loss of generality that 
of is a non-increasing for / > 0, and for some fixed 5 let V be the minimal index such that 

oo 
l=L'+l 

Let Zs = Ylii=i '^i^i- Using (|53] ) and convolution of pdfs, 

^og^<e^'(A^) , 



n - fz',{z) 
where Z'^ is AWGN with the same variance as Z^. Since e^' approaches zero as a function of K, 

lim Pr{Zo + ZsiVK} = ^ , 

for lattices which are good for AWGN coding. 

We are left with the "tail" Z = Y^'^L'+i ^i^i^ which has variance 5. By continuity arguments, 

lim Pr{Zo + Zs + Z ^ Vk\Zo + Z5 € Vi^} = . 

The result follows now by standard arguments of taking e and 5 to zero simultaneously. We have 
assumed the use of a sequence of lattices that is simultaneously Rogers-good and AWGN-good. By Q, 
such a sequence indeed exists. 
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B. Proof of Lemma [7] 

By the properties of the modulo-lattice operation, X„ is a white process. Now the channel from X„ 
to Yn is identical to the channel of (l25l) . thus we have that: 



where Z„ has spectrum S^{e^'^'^^) (1331 ). and consequently Z" = Z„ * ((5.„ —pcn) is its white prediction 
error, with variance ^—^6c according to (|27] ). Now since y„ = y^ — /„ is the optimum linear estimator for 
Xn from the channel output, the orthogonality principle dictates that the estimation error is uncorrelated 
with the process Y^, resulting in an additive backward channel (see e.g. 135]): 

Xn = Yj^ — In + Z„ . 

Switching back to a forward channel, we have 

Y; = aiXn + Z^) + In , 

where Z'^ is white with the same variance as Z". Furthermore, since Z'^ is a function of the processes 
{Xn} and {Zn}, it is independent of all Un- 

C. Proof of Lemma ^ 

By the properties of the modulo-lattice operation, 

-Ml = p{Un ~ Jn) + Zeq^ , 



resulting in the equivalent channel of Figure [80 By the correct initialization assumption, ( 1391 ) holds for 
all past instances, thus T„ is a combination noise (see Definition O. In light of Proposition |2l it is only 
left to show that the variance of T„ is strictly less than the lattice second moment 0c- To that end, 
note that under the correct initialization assumption, the past samples of the process Vn indeed behave as 
samples of a stationary process of spectrum Sv{e^^'^-^) (l32l) . for which Ps{e^'^'^^) is the optimal predictor. 
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It follows that Un — Jn is white, with variance 



Var{C/„ - 


-Jn} = 


Var{C/„|K-i,K-2,.. 


•} 




(a) 


Pe[bU+ ^2 ) p2 




= 


^e(^. + >.)->. 




< 


/32 


"^e(5c7 + 0s)-eS 






(1) 


Pi 

/32 


1 - a 





_ aOc 

where (a) holds by (ITOl ). and (6) holds by applying the same in the opposite direction, combined with 
(l23l) . By the whiteness of ^e(j„ and its independence of all [/„, we have that Un — Jn is independent of 
Zeq , thus the variance of T„ is given by 

Var{T„} = /32Var{C/„-J„} + Var{Ze,„}<^c • (54) 

The margin from 0c depends on the margin in the inequality in the chain above, which depends only on 
Suie^'^'^-f), 9c and (3, and is strictly positive for all (3 < /3o- 

D. Proof of Lemma \3\ 

We work with the optimum Analog Matching encoder for the noise spectrum Szo{e^'^^^). At the 
decoder, we note that for any choice of the channel post-filter G2{e^'^'^^), we have that the equivalent 
noise Zf.q is the noise Z„ = y„ — Xn passed through the filter 1 — Pc{e^^'^^). Consequently, this noise 
has spectrum: 

The filter G2{e^'^^f) should, therefore, be the Wiener filter which minimizes S^ie^'^'^^) at each frequency. 
This filter achieves a noise spectrum 

^^ ^~ Oc- Szo{e^^-n + Szie^^-ff'^'' > 
inside J^c^ and 9c outside. Denoting the variance of the (white) equivalent noise in the case Szo{e^'^'^^) = 
Sz{e^^^f) as So = {l- a)9c (EB, we find that: 

So9c 



|l_P^(e^-2-/)|2 



{9c-Szo{e^^^n)Szo{ei^^f) 
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inside Tc, and Sq/Oq outside. We conclude that we have equivalent channel noise with spectrum 

,,,, Szje^^^f) Oc . Ssje^^^f) - Os 

'"^ ' ^zole^'^'^O '%-5zo(e-''2-/) + 5^(6^2-/)"° ^{e^^-f)0s ° ^ ^ 

inside Tc^ and ^o outside. Now, since this spectrum is everywhere upper-bounded by 5*0, we need not 

worry about correct decoding. The source post-filter input is the source, corrupted by an additive noise 

Zeq^//3, with spectrum arbitrarily close to 

Segje^^^f) _ Ss{e^^-'f)-es 

inside Fc^ and Og outside. Now again we face optimal linear filtering, and we replace the source post-filter 
F2(e-'2'^/) by the Wiener filter for the source, to arrive at the desired result. 
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